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OBSERVATION VERSUS EXPERI- 
MENTATION! 

In gatherings of scientific men such as this 
one, it is customary to have a number of non- 
technical addresses, which often take the form 
of general surveys of certain fields of science, 
with summaries of what is known in various 
directions, and with indications of problems 
which await solution. The topic which I have 
chosen, however, would indicate that for the 
moment it seems to me worth while to stop 
and discuss somewhat the methods of science 
rather than the results. No doubt all of us 
look upon both observation and experimentation 
as necessary evils, the means to arrive at ends 
or ( results which are much more important and 
attractive in themselves than are the processes 
of obtaining them. 

Before a company of astronomers the eon- 
test between observation and experimentation 
might be anticipated to mean a discussion of 
the relative merits of the old and new astron- 
omy, the astronomy of position, or of precision 
as its devotees often call it, and the newer field 
of astrophysics. Or the contest might be be- 
tween the whole field of astronomy on the one 
side and the domain of physics and other ex- 
perimental sciences on the other, for we astron- 
omers have the reputation of being precise 
and painstaking observers, while the experi- 
menters, have, to our minds at least, the habit 
of spending most of their energies in getting 
ready to be precise, and then when they are 
prepared to take what we would call observa- 
tions, their aim is achieved and they pass on 
to something else. But my purpose is rather 
to consider somewhat the struggle which often 
goes on in the mind of the investigator him- 
self, whether he shall after a certain amount of 

l Address of the retiring vice-president and 
chairman of Section D — Astronomy, American 
Association for the Advancement of Science, 
Toronto, December, 1921. 
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preparation begin observing, or whether he 
shall consider that his conditions are not yet 
favorable for exaet work. Likewise the ques- 
tion may come up in any long series of ob- 
servations: When is it better to stop and try 
to improve things rather than to go on in a 
routine? A similar choice may come to an 
individual even in deciding his preference for 
one science or another, and in the fundamental 
sense this same choice runs through much of 
our lives, the attraction of the old versus the 
new. 

While we may take up certain considerations 
from the limited point of view of astronomers, 
there are undoubtedly applications of these 
same ideas in many fields of science. Of 
course we are all interested in improvements, 
and no one of us would care to admit that he 
has not the patience and concentration to keep 
at a task until he has mastered it and can do 
it well. There are, however, differences in indi- 
viduals, and as time goes on these are accen- 
tuated, and each worker naturally tends to 
gravitate into the field where he works best 
and feels at home. The skillful observer is 
usually an orderly person who keeps his sur- 
roundings and apparatus neat and tidy. His 
instinct is to maintain constant conditions, 
and if his instrument or apparatus is working 
perfectly, to let everything remains undisturbed. 
There is good reason for this, since experience 
often shows that variation of conditions intro- 
duces unsuspected errors. The experimenter, 
on the other hand, seems to take delight in 
being surrounded by the debris of his work. 
Order and system are not part of his creed. 
He has no hesitancy in dissecting any fine new 
instrument if some of its pieces will fit in with 
what he wants, probably much to the consterna- 
tion of his colleague who is responsible for the 
•equipment. Whenever the observer sees or does 
anything, he writes down a note, but writing is 
the last thing of which the experimenter 
thinks. The observer takes apparatus as it 
«omes to him, the experimenter improves ap- 
paratus or devises something new. The ob- 
server keeps all or almost all his work, the 
experimenter has no scruples in throwing away 
.anything which he thinks he can improve upon. 

I remember visiting a laboratory in company 



with a prominent astronomer, where we were 
shown some spectrum photographs. The 
physicist in charge showed us a negative which 
he had just taken, and then threw it aside. My 
companion promptly asked if there could not 
be something of value on that plate, if it 
should not be kept. The experimenter an- 
swered that he had dozens equally poor, and 
that he could reproduce it at will. To the 
observer even a poor photograph may repre- 
sent an opportunity which will never return. 

It is much easier to teach large classes of 
students to observe, after a fashion, than to 
experiment. In a laboratory section, the stu- 
dent will consider favorably a system which 
enables him to come in and sit down at his 
table, and without delay to begin and simply 
take readings. We hear a great deal about 
teaching the scientific method, but it would be 
quite impracticable to inflict upon elementary 
students the real methods of science, the trials 
and waste of time which any one must undergo 
before he can determine what he needs, and 
then find and assemble his apparatus. 

There is one direction in which an observer 
sometimes feels that he has the advantage over 
the experimenter, and that is in this matter of 
waste of time. An hour's work for the ob- 
server brings an hour's results, whereas the 
experimenter often puts in a great deal of 
effort with apparently no return. A safe pro- 
gram of observation brings in sure returns; 
but is not any one mistaken in assuming that 
he can avoid waste of effort? It is the fate of 
most scientific work to be superseded, and the 
most accurate observations are likely to be 
quite out of date even in the lifetime of an 
individual. Bradley's star places have been 
and are still of great importance as a basis for 
proper motions of stars, but the time will come 
when the so-called modern observations will be 
of the same order of antiquity as those of a 
century and a half ago, and Bradley's observa- 
tions will gradually lose their importance. On 
the other hand, there are many results from 
positions and proper motions of stars deter- 
mined from current measures which are obvi- 
ously of permanent value. Such a case is 
Boss's cluster in Taurus, a group of stars now 
widely dispersed, but which as time goes on 
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will gradually condense and become more and 
more conspicuous as an illustration of what 
can be predicted from precise data. 

Photographic parallax determinations seem 
to be relegating all previous results to the 
discard, but parallax observers might as well 
hurry and get these results while they are still 
valuable, as the spectroscopic method though 
at present dependent upon the trigonometric 
results for a basis need not always remain so, 
and the mere possibility of interferometer 
measures of parallaxes should be enough to 
dampen one's enthusiasm for undertaking too 
large a program of safe and sane trigono- 
metric determinations. 

One of the striking differences between ob- 
servers and experimenters is their use of the 
method of least squares. I have heard a young 
physicist state that he had been advised against 
taking a course in least squares, because he 
would never have occasion to use that subject 
in physics. The answer is that both he and 
his adviser have probably used the method a 
great deal, without being aware of it. Experi- 
menters as a rule do not repeat measures 
enough to get many residuals — one astronomer 
has said that he wants at least fifty observa- 
tions to determine a reliable probable error — 
but the method of least squares is by no means 
as limited in its usefulness as might be 
imagined. It is striking in how many fields 
of exaet science the discussion of measurements 
takes the directions of a graphical exhibition of 
the results. The experimenter gets some 
measures which he puts on a graph exhibiting, 
say, the dependence of one variable upon an- 
other. Through a series of plotted points he 
proceeds to draw a curve; but how does he 
draw this curve Just what does he try to do 
when he makes a smooth line pass through a 
series of points Even for the simplest case of 
a straight line if you ask a student what he 
does, he may say that he tries to draw the line 
as "near as possible to all of the points," what- 
ever they may mean, or he may try to have as 
many points on one side as on the other side 
of the line. It is very doubtful if by intuition 
he will draw that line which makes the sum of 
the squares of the residuals a minimum, and it 
is difficult to see how he is to fit any curve to 



observations without using some of the prin- 
ciples of the method of least squares. 

In passing it might be noted that some 
authors still persist in publishing curves with- 
out representing the observed points on which 
these curves are based. Such a suppression of 
evidence should not be countenanced, especially 
as the graph of the original observations gives 
any one else such a convenience test of the relia- 
bility of the curves. 

An application of the method of least 
squares which is of the utmost importance to 
the experimenter is in the law of propagation 
of error. The well known relation 
/dX\2 /dX\2 

w = (ss)*+ W*+ • - - 

where B is the probable error of X, a function 
of several measured quantities, X 1? X 2) . . . ., 
is not only useful for determining the probable 
error of a result, but is even more important 
in planning a program of observation or of 
experimentation. Where several quantities 
enter into a determination there is no object in 
spending time or effort in the wrong place, and 
one wonders at the tremendous amount of mis- 
directed effort which is constantly being 
wasted because of investigators measuring and 
being careful about the wrong thing, when an 
elementary acquaintance with this formula 
would show them which of the various sources 
of error was contributing most to the inaccu- 
racy of the result. Another advantage of the 
method of least squares is that it enables a 
number of unknown quantities to be disentan- 
gled from a mass of data where it has been 
impossible for the experimenter to differen- 
tiate with respect to one variable at a time. In 
astronomical practice this is too elementary 
even to mention, but it is amazing how 
physicists and others can get along without 
knowing' how to proceed when the conditions 
are such that they can make only indirect 
observations on several quantities. It is, of 
course, the safest practice to measure directly 
the quantity sought, and to vary but one thing 
at a time when that is possible, but an experi- 
menter may find advantage in knowing how to 
derive several unknowns simultaneously. 

However, with all of the advantages of the 
method of least squares, it is not so seldom 



32 



SCIENCE 



[Vol. IiV, No. 1411 



that its devotees may go too far with it. How 
often it occurs that the accuracy of a series of 
measures as indicated by the probable error is 
illusory In almost every field of exact meas- 
urement we have the presence of both acci- 
dental and systematic errors, and he is an 
optimist indeed who deals with only the former. 
It is here that the experimenter is at an ad- 
vantage, as he naturally is constantly seeking 
to eliminate undesirable factors, and by con- 
stantly changing conditions may vary or elim- 
inate what may be ealled the systematic errors. 

It has been said that a worker in exact sci- 
ence usually goes through three stages of atti- 
tude toward his work. He starts out by con- 
sidering every small or unexpected discrepancy 
as due to a physical reality; after being de- 
ceived a sufficient number of times, he has a 
reaction, and nothing is proved until it is really 
proved; he then gradually grows back into a 
state where he is neither too exultant at the 
first prospect of a discovery, nor too pessi- 
mistic over the insufficiency of the evidence for 
a result which he hopes to establish. We may 
quote from Langley, who in the discussion of 
small irregularities of his bolometer records of 
the solar spectrum said, "When we approach 
the limits of vision or audition, or of percep- 
tion by any other of the human senses, no 
matter how these may be fortified by instru- 
mental aid, we finally perceive, and always 
must perceive a condition, a condition still 
beyond, where certitude becomes incertitude, 
although we may not be able to designate pre- 
cisely where one ceases and the other begins. 
This is always the case, it would seem, on the 
boundaries of our knowledge in every depart- 
ment, and it is so here." 

In the estimate of the precision of a given 
result there is not yet adherence to the logieal 
use of the probable error as a measure of 
precision or accordance; astronomers long ago 
adopted this usage, but others seem to get along 
without it. Only recently I heard in a public 
address the statement that a certain measure 
could be made "with an error of one part in a 
thousand." Just what was meant by this 
would be difficult to determine, especially as 
the speaker afterwards said that the "range 
did not exceed one pari in a thousand." These 



loose statements did not eome from a beginner 
but from a master in the art of exact measure- 
ment. Still another example is found in a 
recent number of a standard journal: "The 
maximum error is .1 per cent." This is pre- 
sumably some sort of estimate of the possible 
systematic error of the res.ult, but one would 
think that physicists would eome to some 
common ground in describing their errors, so 
that they could understand each other. One 
suspects that here we have simply an illustra- 
tion of the difference between the observer and 
the experimenter; the former stays with his 
measures long enough to have a real basis for 
computing a probable error, the latter has a 
few measures, and even if he used the formula 
for the probable error he would be doubtful 
of its value. Experimenters boast when they 
have achieved "astronomical precision" in the 
number of significant figures in their results, 
but they might equally well cultivate some 
astronomical accuracy of statement when it 
comes to describing the reliability or accord- 
ance of their results. 

The term "astronomical" precision brings to 
mind the prediction of some years ago that 
most new discoveries in physics would be in 
the sixth place of decimals. Whatever else 
may be said concerning the advances in that 
science, it will not be maintained that so many 
significant figures have been necessary to estab- 
lish the important results. Intelligent lay 
opinion might be somewhat shocked to learn 
by what methods astronomers are measuring 
or estimating distances of stars. A mere guess 
at the mass of a stellar system may give its 
distance with far greater accuracy than could 
possibly be secured by the method of exact 
measurement. The new things in seienee con- 
tinue to be not in the last place but often in 
the first place of decimals. We should be quite 
happy to have one significant figure correct in 
a measure of the size of the visible universe. 

There is one particular field in astronomy 
where the technique of observing as at present 
practiced is a constant reminder to the ob- 
server that either he or some one else had 
better do some experimenting, and that is in 
astronomical photography. Many an observer 
during the tedious hours of long exposure 
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must have felt that some of his time might 
better be devoted to increasing the sensitivity 
of the photographic plate, rather than to be 
continuing the drudgery of keeping a telescope 
accurately on a star for hours at a time. How- 
ever, the astronomer knows well that the plate 
makers themselves are fully alive to the desira- 
bility of faster plates, which would have such 
an enormous commercial value that the astro- 
nomical applications would seem trivial in 
comparison. Nevertheless, one can not but 
speculate on the field which would be opened 
to small telescopes if the photographic plate 
were increased say tenfold in sensitivity, not 
to mention the power which would then come 
to large instruments. 

There is little need of discussing the relative 
advantages of large and small telescopes, one 
might as well discuss the possibilities of abun- 
dant and meager resources ; but there is at least 
the consolation to a possessor of a small instru- 
ment that he does not need to use it all the time 
simply to justify the capital expenditure in his 
equipment. He is therefore much freer to try 
out new ideas, and even to waste a great deal of 
time, without the immediate necessity of pro- 
ducing results in proportion to his facilities. 
The large and well equipped institutions have 
by no means a monopoly on revolutionary 
improvements or discoveries. 

The choice of an individual between joining 
a large or a small institution may or may not 
be the same as the choice between observation 
and experimentation. In some large places he 
may become simply a cog in the machine, and 
easily sink into a narrow routine. On the 
other hand, the resources of a large place may 
make it possible for him to try out various 
schemes which would be quite impossible if he 
were off by himself. On the whole, one must 
balance the advantages of each type of institu- 
tion, but he is a fortunate individual if he has 
free choice in which direction he will work. 
There is one resource, however, which is neces- 
sary to all scientific investigation, and this is 
the item of time. You may deprive the inves- 
tigator of much of his physical equipment and 
resources, and with plenty of free time he can 
go on, almost with bare hands as it were; but 
take away the opportunity to make continued 



effort, and he will cease to produce. As an 
illustration of what may be done with almost 
no equipment we may cite the case of the late 
Simon Neweomb, who while visiting at a 
summer resort made a determination of the 
fundamental quantity, the total light of all 
the stars. His apparatus comprised only sev- 
eral spectacle lenses, but he succeeded in ob- 
taining a result, and any possessor of a large 
telescope would be satisfied if he could with 
all his means occasionally produce something 
as valuable as that work of Professor New- 
comb. 

But after all, both the experimenter and 
observer need to discuss their work, and this 
entails a certain amount of computation. As 
a rule the observer becomes more adept in the 
art of computation simply because he has more 
of it to do, but either observer or experimenter 
will probably look upon long computations 
simply as necessary evils. It has been said of 
a certain astronomer that his dream of heaven 
is a sky full of comets and a room full of com- 
puters to work out their orbits for him. This 
reminds us that most important of all is the- 
orization; all of the routine of scientific work, 
experimentation, observation, and computation 
are simply a means to an end. The real joy 
consists in sitting at one's desk and making 
discoveries which come out of previous work, 
either from one's own or from that of others. 
Perhaps the ideal case is where a single indi- 
vidual is able to partake in all phases of inves- 
tigation, from the preliminary securing of data 
up to the final discussion of the theoretical 
bearing of the results. In the old days this 
was more easy to do than now, for as science 
becomes more and more complex it is increas- 
ingly difficult for one person to master the 
technique of all the processes involved in a 
single problem, and with the growth of co- 
operative research it is possible for several 
workers to join hands and accomplish what 
would be far beyond the powers of any one of 
them. But in any cooperative scheme it should 
be borne in mind that what is wanted is real 
cooperation on a democratic basis, and not a 
direction of individuals by a so-called master 
mind. Efficient as an autocratic system may 
be, in science as in other fields it ultimately 
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fails in the question of morale, for when young 
scientific workers see that however attractive 
may be the places of the men at the top, the 
chances for any individual are that he will be- 
come only a part of an efficient machine, then 
a man of ambition will choose some machine 
where the material rewards are greater than 
in science. 

One great disadvantage in the arrangement 
of separating the observer and the computer is 
that a realization of attainable accuracy is 
likely to be lost. It sometimes seems that the 
farther the computer is removed in time and 
place from the original observations, the 
greater is the accuracy which these observations 
take on. A good illustration is in some modern 
computations of results based upon old 
observations of variable stars. The method 
of Argelander, of simply looking first at 
one star and then at another, and esti- 
mating the difference of brightness, is still 
of the utmost value, but errors as great as 
ten or twenty per cent, in the ratio of the 
light of two stars are not uncommon. We 
can make the accuracy seem greater by express- 
ing the estimate in stellar magnitude, when the 
errors are only one or two tenths of a magni- 
tude, but the fact remains that the discordances 
are a large fraction of the quantities sought. 
Some computers taking results of such estimates 
have managed to derive elements of variable 
stars where some of the derived quantities are 
expressed to five significant figures, although 
the original data were often wrong in the 
second figure. This fictitious accuracy seems 
to come from a state of mind where the more 
you compute the more figures you get, and the 
investigator needs the restraining influence of 
experience in securing observational data. Of 
course, the computer, if he goes about it in the 
right way, can really show the observer just 
how accurate the measures are, but in his 
anxiety to establish some fine theory the com- 
puter sometimes loses his own sense of pro- 
portion. 

And so it goes; the observer does not know 
how to observe unless he realizes the value of 
experiment; the experimenter loses a great deal 
if he has not acquired the technique of ob- 
servation; neither, the experimenter nor the 



observer can work to the best advantage unless 
he has the proper theoretical background; and 
the pure theorist may be saved from various 
grotesque mistakes if he becomes acquainted 
with some of the methods and difficulties of 
securing the facts of physical science. 

We may, therefore, best dwell not on the 
differences among experimenters, observers, 
and theorists, but rather on their strength when 
united and working together. No matter how 
well rounded an individual may become, his 
capabilities may be easily surpassed by a group 
of cooperating workers. If it be objected that 
new ideas will not originate in a committee, the 
answer is that any one of us has plenty of ideas, 
many of them fundamental and important, but 
what we lack is the ability and power to put 
our ideas into execution. It is here that te> 
my mind lies the great advantage of the policy 
of the National Research Council in bringing 
together in committee workers from all over 
the country so that they can form plans of 
joint attack on various problems. In our uni- 
versities and other institutions there is great 
opportunity for cooperative effort between col- 
leagues, but even in the same institution or 
department the interests may be so divergent 
that a worker may find little help of just the 
kind that he needs, whereas in some other parts 
of the country may be one or more competitors 
who, if they can be got together to talk things 
over, will turn out to be only hearty col- 
laborators. 

Astronomy is called the oldest of the sci- 
ences; our friends in other fields say that it has 
been in the lead in America, and especially 
that astronomers were the first to organize co- 
operation in research. Let us not fail to coa- 
tinue to deserve this good name, and to set the 
example in so far as we can of free trade and 
mutual good will in the solving of our prob- 
lems. 

Joel Stebbins 
University op Illinois Observatory 
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